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Abstract 

In this paper, we study the L 2 functions on U(2n)/0(2n) and Mp(n,R). We relate 
them using the oscillator representation. We first study some isometries between various L 2 
spaces using the compactification we defined in [BJ. These isometries were first introduced 
by Betten-Olafsson in L 3; 0. We then give a description of the matrix coefficients of the 
oscillator representation uj in terms of algebraic functions on U (2n) fO(2n) . The structure of 
L 2 (U(2n)/0(2n)) enables us to decompose the L 2 space of odd functions on Mp(n,M.) into 
a finite orthogonal direct sum, from which an orthogonal basis for L 2 (Mp(n,R)) is obtained. 
In addition, our decomposition preserves both left and right Mp(n, M)-action. Using this, 
we define the signature of tempered genuine representations of Mp(n, ]R). Our result implies 
that every genuine discrete series representation occurs as a subrepresentation in one and 
only one of (cg) p u;) (g> (<g> 2n+1 ~ p w*) for p with a fixed parity, generalizing some result in [T5] . 
Consequently, we prove some results in the papers by Adam-Barbasch [2 and by Moeglin 
[T7] without going through the details of the Langlands-Vogan parameter. In a weak sense, 
our paper also provides an analytic alternative to the Adam-Barbasch Theorem on Howe 
duality ( [14]). 

1 Introduction 

Harmonic analysis on symmetric spaces often involves analysis of spherical functions and 
spherical representations. In the compact case, Helgason's theorem gives the classification 
of spherical representations and the eigen-decomposition of the L 2 space. In the noncom- 
pact Riemanian case, the spherical functions are fairly complicated. For SL(2,M) or more 
generally groups of real rank one, spherical functions are related to a class of special func- 
tions called the hypergeometric functions ( [12], P2]). To study hypergeometric functions, 
typically one needs to study their series expansions and the differential equations defining 
them. Generally speaking, matrix coefficients of semisimple Lie groups, can be approached 
by Harish-Chandra's power series expansion at "infinity " , or the Eisenstein integral ( [T2] 
[12] )• Holomorphic methods also contribute to the studies of matrix coefficients. 

Oscillator representation first appeared in the papers of Shale, Segal and Weil as a "projec- 
tive "representation of the symplectic group. It is quite different from the representations 
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traditionally being studied by people like Gelfand and Harish-Chandra. Its construction is 
not as direct. In spite of this, the purpose of this paper is to show that, for the oscillator 
representation and its tensor products, the matrix coefficients take a much simpler form and 
can be studied systematically. Furthermore, the matrix coefficients of the oscillator repre- 
sentation connects the analysis on U(2n)/0(2n) with analysis on Mp(n,R). Our approach 
also allows us to analyze Howe duality in a way that is different from those of Moeglin, 
Adam-Barbasch ( [14], [17], W- 

Let T n be the Fock space ( PQ). Let V n be the space of polynomials in n complex variables. 
Then V n C T n . Let (uj, T n ) be the Segal-Bargmann model of the oscillator representation 
of Mp(n,R) ( [H], 0). Put 

A(g) = (uj(g)l(z),l(z)), (I) 

where l(z) is the constant function 1. Let u,v G V n . Then M L0 [u®v)(g) = (uj(g)u,v) is 
called a matrix coefficient of uj, with respect to V n - It can be easily shown that M ^®^)(ff) 
descends into a function of Sp(n,R). Throughout this paper, the function M w (it <S> v)/A 
will always be regarded as a function on Sp(n,R). It turns out this function has intriguing 
algebraic and analytic properties. 

In 0, we define an analytic compactiflction Ti : Sp(n,R) — » U(2n)/0(2n) using the 
Segal-Bargmann kernel. By [S], this analytic compactification can be realized as follows. 
Let Pin be the Siegel parabolic subgroup of Sp(2n,M). Let X = Sp(2n,R)/ P2 n - It is clear 
that X = U{2n)/0{2n). Let Sp(n,R) x Sp(n,R) be diagonally embedded in Sp(2n,R). 
Then Sp(n,R) x Sp(n,R) acts on X. In [S], we prove that there is a unique open and 
dense orbit X in X and furthermore X = Sp(n, M). The identification of X with the 
image of 7i is given in the Appendix of [SJ. I should mention that our compactification 7i is 
included in the list of a more general set of compactifications called causal compactifications 
(see Page 294 [3J. So the compacifications I defined in [9] have some overlap with the 
causal compactifications in Betten-Olafsson's list on Page 294 [3]. 

Let / be a function on Sp(n,M.). Let /° be the push- forward of / from Sp(n,M) to 
U(2n)/0(2n). In this paper, we prove two results concerning this push forward. 

Theorem 1.1 (Theorem 5.1 |3j) The push forward f — ► /° defines an isometry be- 
tween L 2 (Sp(n,R), \K{g)\ in+2 dg) and L 2 (U(2n)/0(2n)). 

Here \ A(g) \ is regarded as a function on Sp(n, R). This theorem is equivalent to Theorem 5.1 
in [3] for the group Sp(n, R). The second result states that the push- forward of M "(^jP^Hg) - g 
an algebraic function on U(2n)/0(2n) and it is contained in ©i>oC(o,o,...,0,-2i)(^(2n)/0(2n)) 
(see Theorem I4.1j) . 

Let us fix more notations before we state more results. Let e be the nonidentity element 
in double covering of the identity in Sp(n,R). We call a representation it of Mp(n,M.), 
genuine if 7r(e) = —1. We say that a function on Mp(n,R) is odd if f(eg) = —f{g). Let 
w p,g = (® p w) (8) (0 9 o;*). Let uj c be the real representation uj equipped with a conjugate 
linear multiplication. Since uj is unitary, uj* = w c . Let g = (<3 p P n ) ® (<8> 9 'P^)- Using the 
compactification 7Y, we give a description of the matrix coefficients of uj p>q . 
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Theorem 1.2 The push-forward of (LO P:q (g)u,v)/ A P A spans the space 

®xe2 Sq , p C x (U(2n)/0(2n)), 

where 

S q , P = {(Ai > A 2 > A g > . . . > > Ai > . . . > Ap) | Ai, e Z} 

for p + q < 2n. Here C\(U(2n)/0(2n)) is a space of algebraic functions on U(2n)/0(2n) 
that is equivalent to the irreducible representation ofU(2n) with highest weight A. See 3.4 
for the definition of S q ^ p for p + q > 2n + 1 . 

In connection with the eigen-decomposition of L 2 (U(2n)/0(2n)), Theorem 11.11 and Theo- 
rem 11.21 imply the following. 

Theorem 1.3 (Orthogonal Decompositions) Let M. P: q be the linear span of matrix co- 
efficients of uJ Pt q with respect to V Py q. We have the following orthogonal decompositions 

L 2 _(Mp(n,R)) = © p oddM p ,2n+i-p, L 2 _(Mp(n,R)) = ® p even M Pt 2n+i- P - 
Furthermore, this decomposition preserves both the left and the right Mp(n, R) action. 
The space A^o,2n+i, a Hardy space, is studied in [SJ. 

Let H 2 _(Mp(n,R)) be the set of equivalence classes of irreducible tempered genuine repre- 
sentations of Mp(n,R). Let H 2 q (Mp(n,M)) be the set of equivalence classes of irreducible 
representations that are weakly contained in ujp^n+i-p- In this paper, we prove that 

Theorem 1.4 

n£, 2n+1 _ p (Mp(n,R)) nll 2 pu2n+1 _ vl (Mp(n,R)) = 
if p = pi (mod 2) andpy^pi. Furthermore, 

Il 2 _ {Mp{n, R)) = Up odd U 2 p<2n+1 _ p {Mp{n, R)), U 2 _ {Mp(n, R)) = U p even n 2 ^ 2n+1 _ p {Mp{n, R)). 

Let 7r be a genuine tempered representation of Mp(n,R). Now we define the signature of 
7T to be (p e ,2n + 1 ~p e ,p°,2n + 1 - p°) if 

* G n^ >2n+1 _ p e(Mp(n,R)) nI^. i2B+1 _ p .(Mp(n,R)), 

where p e is even and p° is odd. Our definition of signature is closely related to the Howe 
duality [14) . It can be shown that a tempered genuine representation n has signature 
(p e ,2n + 1 — p e ,p°,2n + 1 — p°) if and only if it occurs in lZ(Mp(n, R), lo) for the dual pair 
(Sp(n,R),0(p e ,2n + l-p e )) and for the dual pair (Sp(n,R), 0(p°, 2n+ 1 -p°)). However, 
the connection is not obvious. Our H 2 2n+1 _ p (Mp(n,R)) only includes irreducible repre- 
sentations that occur in the weak closure of (® p lo) ® (® 2n+1 ~ p {u*)), while lZ(Mp(n,R),oj) 
refers to those irreducible representations that occur as quotients in ((g) p w) ® {® 2n+1 ~ v {tu*)) 
infinitcsimally. 

In [15], Kashiwara and Vergne proved that the representations occur in ® 2n+1 u> are holo- 
morphic discrete series representations of Mp(n,M). From Theorem 11.41 we obtain 
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Corollary 1.1 Fix a parity of p. Every irreducible genuine discrete series representation 
occurs as a subrepresentation of 

for a unique p. 

The pair (p, In + 1 — p) is exactly the signature. For p — 0, In + 1, the intertwining 
operator from (<8> p w) <g> (<g) 2n+1 ~ f 'ui*) to each discrete series representation can be computed 
and explored effectively. This has been done in |15j . For p ^ 0, 2n + 1, the intertwining 
operator 

(® P w)® (® 2 " +1 - p (w*)) ^7T 

is not easy to describe. Our approach does not provide means to compute the signature of 
a discrete series representation. The question of obtaining the signature of a discrete series 
representation can be found in [2 . In fact, Adam and Barbasch proved 

Theorem 1.5 (Thm. 3.3, 5.1 [2j) Fix a parity of p. There is a one to one correspon- 
dence between the discrete series representations of SO(p, In + 1 — p)(0 < p < 2n + 1) and 
genuine discrete series representations of Mp(n,M,). There is a one to one correspondence 
between the disjoint union of the admissible duals of SO(p, In + 1 — _p)(0 < p < 2n + 1) and 
the genuine admissible dual of Mp(n, R). 

In this paper, we prove the discrete series duality without going through the details of 
if -types. We also prove 

Theorem 1.6 Fix a parity of p. There is a one to one correspondence between the irre- 
ducible tempered representations of SO(p, 2n+l— p)(0 < P < 2n+l) and irreducible genuine 
tempered representations of Mp(n,M) . 

In essence, our theorem is a L 2 Howe duality for (0(p, 2n + 1 — p), Mp(n, K)). 

2 Compactification of Sp(n, R) 

The computations in this section are not new. They have been done by Betten-Olafsson in 
a more general setting [3]. 

2.1 Setting 

Let V be a 2rt-dimensional real vector space. Let {ei,e2, . . . ,e2«} be a basis for V. Let 
J = ^ ^ Q™ J ■ W e equip V with the following structures: 

1. the standard symplectic form f2(it, v) = u l Jv; 

2. the standard real inner product (it, v) = u v. 

3. the complex structure 

e n+i = -ie l (i = 1,2, .. .n). 

We denote V equipped with the above complex structure by V c . We equip V c with the 
standard complex inner product (,) . Then 

(it, v) c = (u, v) + iSl(u, v). 
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Let Sp(n,M) be the symplectic group preserving f2(, ). Let 0(2n) be the orthogonal group 
preserving (,). Let U(n) be the unitary group of (V c , (,) c ). Then U(n) = Sp{n, E) n 0(2n) 
and U(n) is a maximal compact subgroup of Sp(n,M). 

Put 

a = {diag(Ai, A 2 , . .., A n , -Ai, -A 2 , . .., -A„) | A; G E, i G [l,n]} 

Then a is a maximal split Cartan subalgebra of sp(n,E). Let A be the analytic group gen- 
erated by a. Let K° be the opposite group of K. Then Sp(n, E) has a KAK decomposition 
and K x K° acts on Sp(ra, E). 

Let S 2n be the space of 2n x 2n symmetric unitary complex matrices. S 2n can be identified 
with U(2n)/0(2n) as follows 

J :geU(2n)^gg t eS 2n . (2) 
The group {/ (n) x {/ (n) acts on S 2n as follows: 

T(ki,k 2 )s = diag(fci, k 2 ) a diag(fc*, k\) {k u k 2 G J7(n)). 



Theorem 2.1 (Analytic Compactification of 5p(n,E), |6j) Let K — U(n). For g 
kiexpHk 2 G ^(n, M), /et 



TL{ki exp i/fc 2 




tanhF c -iscchi? \ ( K 1 
-isecLfl* tanh7? c M k 2 



where H = (Ai, A2, . . . , A„, -Ai, -A 2 , . . . , — A n ) and H c = (Ai, A 2 , . . . , A„). TTiera 
7i is a well-defined map from Sp(n,M.) to S 2n ; 

2. TL is an analytic embedding; 

3. the image ofTL is open and dense in S 2n ; 
4- Identify K x K° with K x K by 



(fci,fe 2 ) -> (fci,fe 



2 )■ 

Then TL : Sp(n, E) — > 5 2 „ is K x K° -equivariant; 

5. Let f be a matrix coefficient of a unitary irreducible representation of Sp(n,R). Then 
f can be pushed forward to a continuous function f on S 2n . 

In short, TL is an analytic compactification of Sp(n,WL). For an arbitrary function / on 
Sp(n,M), we will use /° to denote the push-forward of / to TL(Sp(n,R)). If /° has a 
continuous extension over S 2n , then the extension must be unique. In such a case, /° will 
be used to denote the unique extension. See [6] for details. 
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2.2 Invariant Measure on U(2n) with respect to the Generalized 
Cartan Decomposition 

Generalized Cartan Decomposition is studied in Heckman and SchlichtkrmTs book pTj . 
Unlike the Cartan decomposition which is associated with a symmetric pair (G,G a ), gen- 
eralized Cartan decomposition is built on a symmetric triple (G, G CT , G T ) with (o~, r) a com- 
muting pair of involutions. We start with the structure theory. 

Define 



a{9)= {o n n -i n ) 9 {o n -i n )^ 9)=a - 

Let U = U(2n), K = 0(2n) and H = U(n) x U(n). Let 

p = {iB | B l = B, B e g[(2n, M)} 

and 

-A 

Then 

£/ T = K, U a = H, I © p = u, fj © q = u. 
Let T pq be the torus consisting of 



t = 



cost/ — tsmt 
-is'm8 cos 9 



where 8 € R™. t vq is a maximal Abelian subalgebra in p n q. Then the generalized Cartan 
decomposition (Theorem 2.6, P. 194, says that every g £ U can be written as a product 
k(g)t(g)h(g) . Moreover, 

dg = J(t)dtdkdh 

with 

j(t)= n i* ot -*~ a r-i* a +*~ a r-- 

Here m+ = dimu+ and m~ = dimu~ are the multiplicities of the complex root space 

u+ = {x £ uc I (tt(x) — x, [h,x] = a(h)x,Vh e t pq } 

u„ = {x e uc | (tt(x) = — x, [/i,a;] = a(h)x,Vh 6 t P9 }. 
This result is due to Heckman-Schlichtkrull. 
Theorem 2.2 for 

_ / cos 6* — zsin( 
I — i sin cos 

^(*) = c|]J(cos46» i - cos 40,) || JJcos20i 

_ffere c is some constant. 
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Proof: Notice 

Ui V 
aT[ ~V U 









H 


v V 1 


u 2 J 



Thus 

u+ = { 



( iV* U 2 ) 1 U * + Ul = ° = ^ + 1/21 ^ ^ 7 6 S [ ( n ' R » 



u ^ = ^( v* u 2 i \ut + Ui = o = uZ + r-'-2. i i..r 2 .re P «>. 



Similarly, 



u- ={( ) r; = r,.n = r,. ri.^i-^ii/i.ci}. 

Let C = -4- f {" 7 \ ^) . Then 

, %e\ r _i_ ( 8 

( 1 i& J ° ~ \ 

/ Z7i V \ x 1 / U x +U 2 + W + V U l -U 2 + W-V 
\W U 2 ) 2\ Ui-^-W + V Ui + U 2 -W - V 

Therefore 

Cu+C- x = {(^ X z Y xt ^)\Y t + Y = Z t + Z = 0} 



Let ei(9) = 9 t . Then 



m ±e z ±e, = m ±e . ±e . =1 (i ^ j) 

™2e 4 = 0;m2 e . = 1. 



By Theorem 2.7, P. 194, [II], 

J(i) =|2 2 " 2 -™ J| cos^ - 0,-) sin(9i - 6j) cos^ + 9 J )sm(9 i + 9j) JJ cos(26>i)| 

=2" 2 1 sin(2^ - 26 j) sin(20i + 29 j) JJ cos(2^)| 

=c| JJcos(4(9i) - cos(46j) JJcos(2^)| 

i>j i 

Q.E.D. 



(3) 
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Under the generalized Cartan decomposition, the map j : U(2n) — » U(2n)/0(2n) (see 
Equ. [2]) becomes, 

cos 6* —isind \ , , / 008 20 — i sin 29 \ , f „ TT , 

■ ■ a a & -> M • ■ r,n oa U G 5 2 „ (h £ H,k £ K). 
—ismO cos 9 J \ —ism2& cos 29 J 

Thus the invariant measure ds for 5 2n is given by J(t)dhd9 for s = ht 2 h f . 
Corollary 2.1 Under the generalized Cartan decomposition 



= h 



cos 9 —isin.9 
-is'm9 cos 9 



the U(2n) invariant measure on iS 2n is given by 

ds = c\ H(cos(26> j; ) - cos(20,-))| \ \\_cos 9 % \dhd9. 

i>3 i 

2.3 The Jacobian l^^l 

Consider the group G = Sp(n, ffi) and the root system 

S(a,g) = {±e i ±e j (i > j);±2e t (i,j G [l,n])}. 
Under the Cartan decomposition g — k\ expiJfc 2 with (fci, fc 2 ) £ x U(n), 

dg =| sinha(ff)|dfcid£; 2 (iA 

QeS+(a, fl ) 

= | J^J sinh(Ai — Aj) sinh(Ai + Aj) sinh 2Xi\dXdk\dk2 

i>j i 



=2 ™ 2 |(cxp(A 4 - Xj) - exp(-Ai + A i ))(exp(A i + Xj) - exp(-A l - Xj))\ 

i>j 

Jl |(cxp(2A l ) - exp(-2A l ))|dfcidfc 2 dA 

i 

=2~™ 2 J||(cxp2A 4 + exp(-2A l ) -exp2Aj -exp(-2A i ))| 

i>j 

JJ |(exp(Ai) - exp(-Ai))(exp(Ai) + exp(-A. t ))|dfcidfc 2 <iA 

i 

Recall from Theorem 12.11 TL\sU (n) x U(n) equivariant and 

„. TT .. , , , . ( tanhA — isechA A / cos# — isin( 
H : exp H = d iag (exp A, exp(-A)) - _ ^ x j = ^ _ . ^ Q ^ Q 

Thus 

cos0 = tanhA; sin fl = sechA. 

It follows that 

— sin9id9i — sech 2 XidXi] cos9id9t = — sechAi tanh XidXi. 



(4) 
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Consequently, 



— — - = — sechAi = — sin^ —-^ — — csc (9i — — coshAi 



We state the following theorem concerning the Jacobian of TL. 

Theorem 2.3 Let ds be a U (fin) -invariant measure on S2n- Let dg be a Haar measure on 
Sp(n,M.). Let g = k± expiffea with H = diag(Ai, . . . , A„, — Ai, . . . , — A n ). Then 

dU{g) = c(]JsechA l ) 2n+1 d.9. 

i 

Proof: For every (i > j), 

(exp(2Ai) 4- exp(— 2Ai) — exp(2Aj) — exp(— 2Aj))(sech 2 AiSech 2 A^) 
exp(2Ai) + exp(— 2Aj) — cxp(2Aj) — cxp(— 2Aj) 



=16 
=16 



(exp Ai + exp(— Ai)) 2 (exp Aj + exp(— A^)) 2 
(exp Ai + exp(-Ai)) 2 - (exp A,- + exp(-A :) )) 2 
(exp Ai + exp(— Ai)) 2 (exp Xj + exp(— Xj)) 



2 



(5) 



=16( 1 1 } 

(expAj — exp(— Aj)) 2 (exp Ai — cxp(— Ai)) 2 

=4(scch 2 Aj — sech 2 Ai) 

=4 (sin 2 9j - sin 2 9,) 

=2(cos26»i - cos 26 j) 

For every i, we have 

sech 2 Ai(exp Ai — exp(— Ai))(exp Ai + exp(— Ai)) = 4tanhAi = cos#i (6) 

- sechAidA,, = dBi (7) 
Multiplying these three equations together and taking the absolute value, we obtain 

JT |(exp(2Ai) + exp(— 2A-) — exp(2Aj) — exp(— 2A J ))(sech 2 AiSech 2 A J )| 

i>3 

|sech 2 A^(expAj - exp(-A,))(exp X. t + exp(-Aj))| |sechA l |dA l ^ 

i i 

=c Y[ I (cos 26», - cos 20j-)| II I cos e M e i 

i>j i 

Since 7i is U(n) x /7(n)-equivariant, from Cor 12.11 and Equation dj we obtain 

c([[sechXif n+1 dg = dH(g). 

i 

Here c is used as a symbolic constant. Q.E.D. 

Corollary 2.2 (See Theorem 5.1 [3]) The push forward f — ► f° defines an isometry 
between L 2 (Sp(n,R), (J\sechXi{g)) 2n+1 dg) and L 2 {S 2n ,ds). 
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3 Functions on S2 n and Sp(n,M) 
3.1 Helgason's Theorems 

Let (U, K) be a reductive symmetric pair of compact type (see [12]). Let tt be an irreducible 
representation of U . tt is said to be spherical if there exists a nonzero vector that is fixed 
by ir(K). Let p be the orthogonal complement of t in u. Let t p be a maximal abelian 
subalgebra of p. Let M be the centralizer of t p in K . Let t be a maximal toral subalgebra 
of u containing t p . 

Theorem 3.1 (Helgason's Theorem) Let tt be an irreducible representation ofU. Then 
tt is spherical if and only if tt(M) leaves the highest weight vector fixed. Furthermore the 
spherical vector is unique up to a scalar. 

Let S = U/K. An immediate consequence of Helgason's theorem is the decomposition 
theorem of square integrable functions on S. 

Theorem 3.2 Let L 2 (S) be the space of square integrable functions with respect to the 
JJ -invariant measure on S. Then L 2 (S) is the closure of 

tb/i spherical^ /i 

Here C M (<S) is equivalent to the irreducible unitary representation with highest weight fx. 

Now let U = U(n), K — 0(n). Then (U,K) is a reductive symmetric pair of compact 
type. Even though Helgason's original theorem assumes that U is simply connected and 
semisimple, it remains valid for (U(n), 0{n)). Let 

t p = {diag(i<M0 2 ,...i0 n ) \9 l eR}. 

Then the Weyl group W(U, K) is the permutation group on {8i}. The centralizer 

M = {e = diag(ei,e 2 , ■ . . , e„) | = ±1} C T p . 

Let V\ be an irreducible representation of U(n) with the highest weight 

A = (Ai > A 2 > . . . > A„). 

Notice that for vq, the highest weight vector of (tt, V\), 

7r(e)« =nr =1 ( ei ) A ^ . 

Therefore V\ has a 7^-fixed vector if and only if every Ai is even. We say that A is even if 
every is even. We obtain 

Corollary 3.1 Consider the reductive symmetric pair (U(n),0(n)). An irreducible repre- 
sentation V\ of U (n) is spherical if and only if A is even. Moreover, we have 

i 2 (5)=0A evenC X (S) 

where C\(S) is an irreducible representation with highest weight 

A = (Ai > A 2 . . . > A„). 
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3.2 The Metaplectic Function 

For a 2n x In matrix g, define 



1 

2 

Suppose that 



C a = -(g-JgJ). 



Then 









S)- 


A+D 


B-C 




2 


C 2 -B 


A+D 


2 


2 



Define 

It is known that 6 GL n (C) when 5 g Sp(n, K) (See [IBJ). One can now write down the 
metaplectic group Mp(n,M) precisely as follows (see [15] ) 

Mp(n,R) = {(A, , 9 ) I .g e Sp(n,K), A 2 det(C^) = 1}. 

Define the metaplectic function A on Mp(n,M) to be A(X,g) = A. Then A 2 = det(C^)" 1 
is a function on Sp(n,R). \A(X,g) = | detC^" |~ 5 is also a function on Sp(n, K). 

Let g = ki exp7Jfc 2 be a ivTAX decomposition of 5p(n,M). Recall from [6] that 

= k r cosh(H c )k 2 

Thus _ 

A 2 ( 5 ) = det(C*^)- 1 = det(fc 2 - 1 sech(i? c )fcf 1 ) = det(/c 2 sech(iJ c )fc 1 ) 

and |A 2 (g) = Yii sechAi(g). Over <S 2 „, define a function 

deti2 : s = ( S t n Sl2 ) 6 S 2 „ — > detsi 2 . 

V S 12 S 22 / 

From our definition of 7i, we see immediately that 

A( 5 ) 2 = det(C^)- 1 = det(fesech(iJ c )fc^) = det 12 (iH(g)). 
So the push-forward of A 2 under H is 

(A 2 )° = l "det 12 . 

Let V be the standard 2n-dimensional Hilbert space with orthonormal basis 

{d, e 2 , . . . e 2 „}. 

Let U(2n) act on V canonically as unitary operators. Then (A n ,A n V) becomes a unitary 
representation of U(2n). In particular, 

deti 2 (s) = (A"(s)ei A e 2 A ... A e n ,e n+1 A e n+1 A ... A e 2 „). 

We summarize our discussion in the following theorem. 

Theorem 3.3 The push-forward of A 2 under Tt is 

i"deti 2 (s) = i n (A n (s)e 1 A e 2 A ... A e n , e n+ i A e„+i A ... A e 2 „). 

where e± A e 2 A . . . A e„ is a highest weight vector of A™ and e n +i A e Jl+2 A ... A e 2ll is a 
lowest weight vector. 
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3.3 An Isometry Between L 2 (S 2n ) and L 2 _(Mp(n,R)) 

Let e be the nonidentity element in the metaplectic lifting of the identity element in Sp(n, R). 
We say that a function / on Mp(n, R) is odd if 

f(eg) = ~f(g) (g G Mp(n,R)); 

we say that a function / on Afp 2 n(R) is even if 

f(eg) = f(9) (~g € Mp(n,R)). 

Let L 2 _ (Mp(n, R)) be the space of odd square integrable functions of A/p(n,R). Let 
L+(Mp(n, R)) be the space of even square integrable functions of Mp(n,M.). We iden- 
tify L\(Mp(n,R)) with L 2 (Sp(n,R). For any / £ L 2 _(Mp(n,M.)), observe that /A 2 ™ +1 is 
an even function. Regarding fA_ 2n+1 as a function on 5p(n,R), we define 

/ : / S L 2 _(Mp(n,R)) - (/A^ 2 "- 1 ) . 

/(/) is a function on S2n- 

Theorem 3.4 (See Theorem 7.1 |3j) With a proper choice of the invariant measure 
on Mp(n,M), the map I defines isometry from L 2 _{Mp(n^)) to L 2 (S2n)- 

Proof: Let f,he L 2 _(Mp(n,R)). Then 

2 (Mp(n,R)) — 

/ f(g)h(g)dg 

JgeMp(n,R) 

= [ (f^~ 2n ~ 1 )(g)(hA~ 2n - 1 )(g)\A 4:n+2 (g)dg 

J Mp(n,K) 

=2 / (fA-^MiUT^Mllsech^X^dg ( 9 ) 

JSp(n,R) i 

=2 / I(f){ S )T(h)(s)d S 
=2(I(f),I(h)) LHS2n y 

Q.E.D. 

3.4 An Orthogonal Decomposition of I/ 2 (iS 2 „) 

Let Z™ be the integral lattice of n dimension. Let 2Z™ be the sublattice of even integers. 
Let det be the determinant function of s £ iS 2 n- The determinant function is a weight 
(—2, —2, . . . , —2) function on iS 2n = U(2n)/0(2n). For p + q < 2n, we define a subset of 
Z 2 ™: 

p 2ji— p—q q 

S p , q = {Ai > A 2 > . . . > Ap > = 0.*.. = > mi > > p,q}. 

For p + q > 2n + 1, we define to S PtQ be the union of S Si t with s < p,t < q and s + t < 2n. 
Write 

2n 

m = (m, m^. . . m); 
Sp^ + m = {A + m|Ae S^g}. 
Let 0(iS 2 „) be the space of regular functions on iS 2rl . 
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Theorem 3.5 

^7p even,p-\-q— 2n+l 

S p , q + P n2Z 2 " (S 2 n) • 
vbp even,p-\-q—2n-\-l (5 2 „). 
0(<S 2n ) = even,p-\-q— 2n+l (<S 2n ). 

Proof: We will only prove the first statement. It suffices to show that for any A even, 
there exists a unique p + q = 2n + 1 with p even such that A G S p _ q + — . Observe that 
A G S 2 i,2n+i-2i + 2z if and only if A 2j+ i < 2i and A 2 j_i > 2i. If Ai < 0, then A G So, 2n +i- 
Otherwise, Ai > 2. If A 3 < 2, then A G S 2 ,2n-i + 2. Otherwise A 3 > 4. If A 5 < 4, 
then A G S4,2ri-3 + 4. We continue on this process. If A 2 ^+i < 2i and A 2 i_i > 2i, then 
A G S 2 i,2n+i-2i + 2z. Finally, if A 2 „_i > 2n, A G ^n.i + 2n. Q.E.D. 

Theorem 3.6 

0(S 2n ) = (s 2n ). 

Proof: Suppose that A is even. Notice that A G S p . q + 2p if and only if A p > 2p > X p+ i. 
If Ai < 0, then A G So, 2n - Otherwise, Ai > 2. If A 2 < 2, then A p > 2p > \ p +i for p = 1. 
So A G 5i j2 „_i + 2. Otherwise, A 2 > 4. If A 3 < 4, then \ p > 2p > A p+i for p = 2. 
So A G 5 , 2 .2n-2 + 4. We continue on this process until p = 2n for which A 2n > 4n. So 
A G S 2 „, +4n. Q.E.D. 

Let F be an I? function on S 2n . Then F has a Helgason-Peter-Weyl expansion 

Aeven 

where F A G Ca(S 2 „). Let 5 be a subset of Z 2 ™. Let L 2 (<S2„)(S') be the subspace of i 2 - 
functions whose HPW expansion only contains F\ with A G S. We have 

Theorem 3.7 

L 2 (S 2n ) — (Bp even,p+q=2n+\L 2 ' {S 2n ){S pjq + pfl 2Z 2 ™); 

L 2 {S 2n ) — (Bq even,p+q=2n+lL 2 (S2n)(Sp,q 2Z 2n ); 
L 2 (S 2n ) = ®p +q =2nL 2 (S2n)(Sp, q + 2p n 2I? n ) . 



4 Matrix Coefficients of The Oscillator Representation 

Let me first introduce the Bargmann-Segal Model of the oscillator representation. Let 
V = C" and . 

dfi(z) = exp(--||z|| 2 )(izi . . . dz n 

be the Gaussian measure. For simplicity, let dz be the Euclidean measure on V. Let V n be 
the space of polynomials on V. We define an inner product on V n 

if, 9) = / fgdfi(z). 
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It is well-known that 

(«V) = (a ±0) 
(z a ,z a ) = 2 a al 
(See [T]). Here we follow the multi- index convention and 



2H |a| = £> 



i=l 

Now let ,F n be the completion of V n . Then J 7 ,,, is precisely the space of square integrable 
analytic functions on V (see [T]). 

For (X,g) £ Mp(n,R), we define 



<»(\g)f(z) = J Xe^{-{i^M)H{g)\ _ \)f{w)dn{w) 

Then (w, J^) is a faithful unitary representation of Mp(n, K) (see [IE]). This model is often 
called the Bargmann- Segal model of the oscillator representation. 

4.1 Matrix coefficient Maps 

Let (f>, ip € P n . Define the map of matrix coefficient 

M u : V n ® 7>„ -» C°°{Mp{n, R)) 

as follows 

V)(<?) = M<?)<^)> V 3 e Mp(n,R). 

We have 

Af w (^ (8 ^)(A, <?) =(w(A, g)</>(z), ^(*)) 

Aexp(-(i2*,uJ*)H(5) f !f j)<fr(w)i[>(z)dfi(w)dfi(z) 



VxV 

1 



^ Aexp(|(z*,w t )H(g) I Z_ J)cf>(w)ip(-iz)dfi(w)dn(z) ( . |(|) 

Aexp(-(z*, W t )H(g) ( 1. \)4>(w)ip(iz)d^(w)d^(z) 
vxv 4 \ w / 

=A(A, 5 ) / exp(i(z 4 ,w 4 M 5 ) ( * ))^H^(i«)d/i(w)d/i(«) 



For any function / 6 T^n, we define a function on 

,1 



W(f)(s)= J^eM^^sl Z _ ))f(z,w)M*)Mu)- (11) 

If we identify V n <8> with T^n by 

j : 4>{w) ® ^(2) — > <p(w)ij)(iz), 
then ® VO) is the push-forward of (g) ■0)(A, g)/A(X, g). 
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Lemma 4.1 Let U(2n) act on C 2 " as unitary operators. Let U(2n) act on S2n canonically. 
Then W is a U(2n) equivariant linear transform from Vm to Os 2n ■ 

Proof: For g G U(2n), g acts on <S 2 n by r(g)s — gsg 1 . We compute 

Wfig-'sig-'Y) = ^exp(l(^,^) 9 - 1 S ( 5 - 1 ) t ( 1 ))/(( * ) )d/x(*)dM«0 
= [ exp(i(z*,w t )s 

Thus W is £/(2n)-cquivariant. Q.E.D. 

Let Mi be the linear span of matrix coefficients of (u,V n )- So Mi C C°°(Mp(n, R)). 
Let Wi to be the linear span of W(f) with / G Vi n . Here 1 is used to indicate that only 
one oscillator representation is involved as we will be discussing tensor products of the os- 
cillator representation later. Notice that Wi is just the push-forward of Mi/ A.. Let V™ n be 
the polynomials of 2n variables of homogeneous degree to. Then it is well-known that 
is an irreducible unitary representation of U(2n) with highest weight (0,0, ... ,0, —to). Wc 
obtain 

Lemma 4.2 Wi is contained in 0ieNC( O ,o,...,o,-i)(52n)- 

Observe that for an odd function f in Vint Wf(s) = 0. One can easily see that for any even 

m, 

w(rz) + o 

since (cj(l)z a ,w a ) ^ 0. Therefore 
Theorem 4.1 

Wl = ©ieNC(0,0,...,0,-2i)(<5>2n) 

In particular, Wi is a subspace of algebraic functions on S2n- Furthermore, Wi is the 
push-forward of Mi /A under H. 

Here we regard Mi/ A as a set of functions on Sp(n,M.). 
4.2 The Contragradient oj* 

Let uj* be the contragradient representation of w. So for S G J-* and u G T ni uj*(g)S(u) — 
5(ui(g~^ 1 u)). By Riesz representation theorem, S can be identified with us G T n such that 

S(u) = (u, u s ). 

It follows that (ujU^.^s) — ui*(g)6(u) — 6(uj(g^ 1 )u) = (ui(g~ 1 )u,us) — (u,ui(g)us). There- 
fore 

U<v*(g)6 = ^(g)u S . 

Let Vn be V n with complex conjugate linear scalar multiplication. In other words, mul- 
tiplication of A G C on / G is just A/. Then wc may define a new representation lu c 
by equipping (w,JT„) with the complex conjugate linear scalar multiplication and the inner 
product 

(u,v) u c = {v,u) u . 



(lhf(g-i( Z w )w(zW(w) 
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Clearly, u>{g) = to c (g). The matrix coefficient 



{uj c (g)u, v) = 0, uj(g)u) = (uj(g)u, v) = M u {u® v){g). 
Observe that uj* = lo c . The identification is given by S — > 715. 

Theorem 4.2 TTie push-forward of M u c(u ® f)/A spans Wi = ©iePiC(2i,o,...,o)(<!?2n)- 
4.3 Estimates on Matrix Coefficients 

Notice that functions in Mi are all bounded functions. Theorem 14. II implies 
Lemma 4.3 For u,v £ 7-V,, |M w (tt®u)| < C UtV \A(X,g)\ for some constant C u>v . 

Here |A(A,p)| = nr=i( secn ^(3))^- For simplicity, write it as |A(g)|. To obtain more 
precise estimates on for / £ V n ® V n and beyond, we prove the following lemma. 

Lemma 4.4 Suppose that f £ L 1 (z, w, exp — i(||z|| 2 + ||u>|| 2 )dzdu;). Then 
\\Wf(s)\\ 

■max z,w),exp — 4 (|| ^ || 2 + ||^ || 2 )dzdw) ' 

Proof: Notice that 

^ '^t —t\ n f z \ \ ^ mm f^- ,"|UI|2 1 ll„„l|2\ 



exp(-(z*,^) a [ * ))<cxp( i (||z|| 2 + || w |j 2 )). 



We have V s £ 5- 
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.1 



C 2 " 



|M//(s)|=| / exp(^(z t ,W t )s( * )/( f, ))^(z)^(«,)| 



< 



^exp(i(||z|| 2 + |M| 2 ))|/(( * ))|e X p(-i(P|| 2 +|H| 2 ))^ 
2 J/(( * ))|exp-i(||z|| 2 + H| 2 )d^) 

I L 1 (2,10 ,cxp — ^ ( || z\ \ 2 + 1| -UJ || 2 )dzdw) 



(13) 



Q.E.D. 



Theorem 4.3 Suppose that u and v are smooth vectors in (w,J- n ). Then there exists a 
constant C n such that 

\\W{u®v){s)\\ max < Cl\\u(z)v(w)(l + \\z\\) n+1 (l + \\w\\) n+1 \\ 

L 2 (z : ™,cxp-i(||z|| 2 + |H| 2 )dzdu;) 

and for every g £ Mp(n, R) 

Proof: Since u is smooth, z a u £ J- n . By Schwartz inequality, 
(/ Kz)|exp4||z|| 2 dz) 2 < / \u{z)(l + \\z\\r +l e^~-\\z\\ 2 \ 2 dz [ |(1 + ||z||) — fdz 
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Therefore, there exists a constant C ra , only depending on n, such that 

MlLUz^xp-idlzlHdz) < + \\ z \\) n+1 \\L 2 (z,cxp-±\\z\\ 2 dz)- 

By the previous lemma, 

\W(u®v)(s)\ < C'2|| u (2 ; )- t ;(«;)(l + HzlD^+^l + || ^ || |L 2(2 ^ !Cxp _ i (^ip + n^ji-^)^^) . 
It follows that 

liv^c-Lt^^Xs-)! < n^ (j2r:) ^ c ^ )(:;L _,_ h^id^+i^! + ||^||)^+ :L || i , 2{; ^^, e3C p_ i . cl|js||2 _ H||w||3)rf ^^ ) |7VC£/)|. 

4.4 Tensor Products of Oscillator Representations 

Let 



p H 
, * v < * 

OJ va ^U!®U!®...®U!®U! C ®UJ C ®...®U! 



P,1 

V p , q = {® p V n )®{® q V c n ). 

Let M. p ,q be the linear span of matrix coefficients of (uj p . q ,V P}q ). Then A / f Pjg /(A p A ) C 
C^Sp^R)). Define 

p p+q 

W p , q = ® P W ® ®«F : A ® / 2 <g> • • • ® f p+q e ® p 7> 2 „ ® ®*7>£„ -» II W W II ^Xft)- 

Let YVp,q be the image of W p<q . According to a Theorem of Kashiwara-Vergne ( [15]). the 
U (2n)-types in {® p Vi n ) ® (^^fn) are parametrized by 

2n-s-t 

(mi > rrt2 > . . . > m 3 > 0, 0, . . . > tt-1 > n 2 > . . . > n t ) 

with s < q, t < p and s + £ < 2n. It is easy to sec that the set of parameters is exactly 
S q ^p. According to Helgason's theorem, the U{2n) types in Os 2n are parametrized by even 
highest weights. We have 

Lemma 4.5 W p . q = ®x e s q , p n2Z^Cx(S 2n )- 

Proof: We only need to show that every C\(S2n) with A € S q , p fl 2Z 2 ™ indeed occurs in 
Wp.q. Let 

2n-s~t 

A = (mi > 777-2 > ■ ■ • > W s > 0, 0, . . . > 77! > 77 2 > . . . > 77j) 

with s < q,t < p. Recall that 

Wl = ©igNC( 0i 0,...A-2i)('S2n) 



Let ?j_2i be the highest weight vector of C r (o,o,...,o,-2j)(»S2n)- Then v\ = IlLi Ili=i v - 
is a highest weight vector in C\(S2n)- Furthermore, v\ is in W Pi9 since 

p p+q 

W p , q = spaniH fi H | fi e Wi V i}. 

z— 1 z— p+1 

Thus C A (5a») C W p , q . Q.E.D. 
By Theorem 14.11 we have 
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Theorem 4.4 The matrix coefficients of lo va can be written as A p A 9 / (H(g)) with f G 
Wp >g . So W Pt q is the push forward of A4 Piq / APAi under H. 

4.5 The function deti 2 
Recall 

deti 2 (s) = (A"(s)ei A e 2 A ... A e„, e„+i A e„+i A ... A e 2 „). 
Let Va be an irreducible constituent of the linear span of 

{det(gsg t )€C 00 (S 2 n) \g€U(2n)}. 

Here A is a dominant weight. Then 

• A is even. 

• V\ contains a vector of weight (—1, — 1, — 1, . . . , —1). 

n 

' * \ 

• A is in the convex hull spanned by permutations of (0, 0, ... , 0, —2, —2, . . . , —2). 

n 

Thus A= (0,0,..., 0,-2, -2*..., -2). 

n 

Lemma 4.6 The function deti 2 (s) G C\{S 2n ) with A = (0, 0, . . . , 0, — 2, — 2, . . . , — 2). Fur- 
thermore, for every s G S 2n , 

det 12 (s) = (-l) n det _1 («)deti 2 (*). 

Proof: We only need to prove the second statement. Consider the generalized Cartan 
decomposition 

_ ( ki \ f cos6 -ism9 \ f k\ \ 
S ~ ^ k 2 ) \ -i sin 6> cos(9 J \ fc* J ' 

The determinant 

deti 2 (s) = det(ihsm6k~2~) = (-l)"(det fci) _2 (det fc 2 )" 2 det(-ih sin6>fc 2 ) = (-1)" det s _1 det 
Q.E.D. 

The function det -1 is regarded as a function on S 2n . It is of the weight (2, 2 . . . , 2) due to 
the left multiplication by g~ x . 

4.6 An Orthogonal decomposition of L 2 _(Mp(n,M)) 

Theorem 4.5 (M p , q ,M Puqi ) — ifpi+qi = p + q = 2n+ 1, pi ^ p andpi = p (mod 2). 
Furthermore 

L 2 _(Mp(n,R) = ©p+ 9 =2„+i,p oddMp, q . 

L 2 _(Mp(n,R) = ©p+ 9 =2n+l,p ewnMp,,- 
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Proof: Suppose that pi + qi = p + q = 2n + 1, p\ ^ p and p\=p (mod 2). Assume that p 
is odd and q is even. For / e M p , q , recall that /(/) = (/A -2 ™ -1 ) . According to Theorem 
13.41 it suffices to show that 

(I(M p , q ),I(M Puqi ))=0 

and 

L 2 {S2n) — ®p+q=2n+l,p oddI{-M-p,q)- 

According to Theorem 13.31 and Lemma 14.61 the push- forward of { — ) q is just 



, 1 x ia ,deti2(s) q . . 

deti 2 (s) 



It follows that 



1(f) = (/A- 2 "- 1 ) = (fA-PA q )°((jr)°. 



By Theorem S3 (/A- p A 9 )° E W P:9 . So 

I(M p , q ) = det(«)-*W Pj ,. 

From Theorem 14.51 

^„, 5 ) = ®Ae5 g , p dets^C A (52„) - ®Aeq+s 9 . p C A (5 2 „). 



Our first assertion follows from Theorem 13.51 Our second assertion follows similarly from 
Theorem EH Q.E.D. 



We denote the closure of M. p , q by cl(M p . q ). Then 

L 2 _(Mp(n,R) = © p+ g=2n+i,p odd,cl{M p , q ); 

L 2 _(Mp(n,M.) = ® p+q =1n+\,p evend(M Pi q). 



Corollary 4.1 (Orthonormal Basis) There exists, B p ^ n +i- P , o, set of elements in 

rp,2n+l-p ® rp,2n+l-p 

smc/i that {M^p 2n+1 _ p {b)}b<£B p 2 n+i-p * s arl orthonormal basis in cl(M Pi 2n+i- P ) ■ In partic- 
ular, all M Up 2n+l p (b) is bounded by a multiple of \\A(g)\\ 2n+1 . 

This Corollary follows from Theorem 14.41 



5 Signature of Genuine Tempered Representations 

Theorem 5.1 Let p + q = 2n+l. Let Mp(n, K) act on L 2 _(Mp(n,M)) from both left and 
right. Then cl(A4 p . q ) is a left and right subrepresentation. Let P p . q be the projection of 
L 2 _(Mp(n,M.)) onto cl(AA P:q ) . Then P p<q is Mp(n,R)-equivariant. 
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Proof: Fix (p, q) . Let B be the linear space spanned by 

u P , q {g)u g e Mp(n,R),u e V PA . 

Clearly, Mp(n, R) acts on B from left. Let Mb be the set of matrix coefficients with respect 
to B. Then Mb is a Mp(n, R)-space. Claim: M p , q C Mb (= cl(M p , q ). 

Clearly M p , q C A^s. To prove the second inclusion, let u,v £ V Pt q. Fix gi 6 Mf>(n,R). 
It suffices to show that g — > (w Ptq (ggi)u, v) is in cZ(A^ Pi g). Since u) Ptq (gi)u is an analytic 
function, u> Ptq {gi)u has a Taylor expansion. Let it^ be its i-th Taylor polynomial in terms 
of the total degree. Since Lo p q {gi)u is smooth, z Q w;^Ui(z)i>(u>) — » z a w^U} P:q (gi)u(z)v(w) in 
L 2 (z,w,dp(z,w)). By Theorem 14.31 

W p , q (ui ® v)(s) -> Wp^(wp !(Z (5gi)M <g> v)(s) 

in sup-norm in C(iS2 n )- In particular, W Ptq (ui (£> v)(s) is uniformly bounded by a constant 
function. Recall that 

\M^ q ( Ul ®v){g)\ = \W p , q (u^v)(n(g))\\A(g)\\ 2n+1 

and |A( 5 )| 2 " +1 e L 2 {Sp(n,R)). It follows that M Up J Ui ®v)(g) -> M^u^G^u® «)(<?) 
in L 2 (Mp(n, R)). Thus, p — > (uj p q (ggi)u,v) is in c/(A^ p . g ). 

Now 5 £ Mp(n, R) preserves on and .Mb is dense in cl(M p . q ). By the unitarity 
of uj P:Q (g), g preserves on cI(Mb) = cl{M M ). So cl(M Piq ) is a subrepresentation of 
Li(Mp(n,l)) under the left regular action. The rest of the theorem follows easily. Q.E.D. 

5.1 Signature of Genuine Discrete Series Representation 

Let D be a genuine discrete series representation of Mp(n,M). Let u, v be infinite vectors 
in D. Then D UiV (g) — (D(g)u,v) is in L?.(Mp(n,ffi)). Now under the decomposition 

L 2 _{Mp{n,R)) = ©p+g= 2 „+i,p oddd(M M ), 

D u ,v(g) decomposes into a sum of functions in cl(M Pl q). What we want to show is that 
D u ,v(g) in actually in one and only one of cl{M p>q ). 

Theorem 5.2 Fix a parity of p and let p + q = 2n+ 1. Let D be a genuine discrete series 
representation of Mp(n,M,). Then there exists a unique p such that the matrix coefficients of 
D are in cl(M Pt q). Equivalently, there exists a unique p such that D is a subrepresentation 
ofcl(M p<q ). 

Proof: Let D be a genuine discrete series representation. Let Hp® Hp be the irreducible 
subrepresentation of Mp(n,M) x Mp(n,M) occurring discretely in L?_(Mp(n,R)). Consider 
P p , q {H D ®H* D ). There exists a (p, q) such that P Ptq {H D ®H* D ) ^ 0. Since P p . q is Mp(n, R) x 
Mp(n, R)-equivariant, P Ptq (Hrj(E>H^ ) ) is a Mp(n, R) x Mp(n,M.) representation. Because of 
the irreducibility of Hd&Hq, P p q (Hrj^H'^ ) ) is equivalent to Hrj^Hp. By Schur's Lemma, 
p P,q\H D ®H* D = xi h d ®h* d - Since F P,i is a projection, P P , q \ HD ® Hl = I Hd ®H' d - So for an Y 
other px + qi =2n+l and pi =p (mod 2), P Pl , 9l {Hd®H* d ) — 0. Therefore, there exists a 
unique p such that Hr>®Ho C c/(A / i Pi9 ). Our assertion follows. Q.E.D. 
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Definition 5.1 Let D be a genuine discrete series representation of Mp(n, R). Fix a parity 
of p. We call (p, 2n + 1 — p) the signature of D for p odd (even) if the matrix coefficients 
of D are in cl(Ai Pt 2 n +i-p)- We denote the odd p by p and the even p by p e . We call 
(p e , 2n + 1 — P(>;p , 2n + 1 — p Q ) the signature of D. 

5.2 Signature of Genuine Tempered Representations 

Let p + q = 2n + 1. Recall that the matrix coefficients of ui p , q are in L 2 (Mp(n,R)). By 
a Theorem of Cowling-Haagerup-Howe [1] , the representations in the weak closure of u) Piq 
are all tempered. Let II? (Mp(n, R)) be the support of uj p , q . Let U^(Mp(n,R)) be the 
set of equivalence classes of irreducible tempered representations with 7r(e) = —1. The set 
IP_(Mp(n,R)) is a discrete union of W 1 /W where n is a nonnegative integer and W is 
a certain Weyl group. The Fell topology on n 2 . (Mp(n, R)) is the natural topology. The 
representation L 2 _(Mp(n,R)) is supported on n 2 _(Mp(n,R)) (see 14.12 [20]). 

Theorem 5.3 Fix a parity of p. U 2 _(Mp(n,M.)) is the disjoint union ofH 2 2n+1 _ p (Mp(n,M.)) . 

Proof: Fix a parity of p. First of all, we want to prove that IPl (Mp(n, R)) = U p H 2 2n+1 _ p (Mp(-r 
Put U = U 2 _(Mp(n,R)) - Uj,h] i2n+1 _ ) ,(Mp(n,l)). Then U is open with respect to the Fell 
topology. If the Plancherel measure of U is zero, then L 2 . (Mp(n,R)) is supported on the 
closure of L) p H 2 2n+1 _ p (Mp(n, R)). Since H p2n+1 _ p (Mp(n,R)) is already closed, we have 

U 2 _(Mp(n,R)) = U p Ul 2n+1 _ p (Mp(n,R)). 

We are done. Now suppose that the Plancherel measure of U is nonzero. Let L 2 _ (Mp(n, R), U\) 
be a nontrivial subspace of L^_(Mp(n, R)) supported on a closed subset U\ in U. Let 
/ be a nonzero function in L 2 _(Mp(n,R), U\). Notice that Lu Pt 2n+i- P is supported on 
lip 2n+1 _ p (Mp(n, R)). It follows that / _L M p ^n+i- P for all p with a fixed parity. By 
Theorem 14.51 f(g) — for all g. This is a contradiction. So U = and 

n»(Mp(n,R)) = U p n£ i2n+1 _ p (Mp(n,R)). 



Next, we want to show that if p\ ^ p 2 and p\ = pi (mod 2) then H pi 2n+x _ pi (Mp(n, R)) PI 
H 2 2 2n+1 _ p2 (Mp(n,M.)) — 0. Notice that Theorem 14.51 only implies that the intersec- 
tion is of Plancherel measure zero. Suppose that (it, H^) £ H pi 2jl+1 _ pi (Mp(n, R)) n 
^■p 2 2n+i- P2 (Mp(n,R)). Let u be a nonzero if-finite vector in i/^. Suppose that u is in 
the if-type a. Consider M^{u ® u). Since 7r is weakly contained in E£ 2 n+i-pi (Mp( n , R)), 
there exists a sequence of elements u, £ Ppi,2n+i-pi such that 2n+1 (tt, ® Uj) — > 

M w (u x u) uniformly over any compact set. We may further assume that Ui are all of 
the if-type a and {||uj||} is bounded. Let p / p\ and p = pi (mod 2). By Theorem 
14.51 M UJpi 2 „ +1 _ P1 (itj <& Ui) J- -M Pj 2n+i-p- Now we want to apply the Dominated Con- 
vergence Theorem to show that M^(u ® u) _L A4 P) 2n+i-p- By a Theorem of Cowling- 
Haagerup-Howe, \M U 2n+i- P1 ( u « ® are uniformly bounded by CS(g) where S(g) is 
Harish-Chandra's basic spherical function 4 . All functions in Ai p ^ n +i- P are bounded by 
|A| 2n+1 . It is easy to see that E{g)\A\ 2n+1 £ L 1 (Mp(n,R)). So by the Dominated Conver- 
gence Theorem, M^{u (g) it) _L -Mp,2ra+i-p f° r all P ^ Pi and p = pi (mod 2). Similarly, 
M,r(u ga) 1 A4 p .2n+i-p for all p ^ Pi and p = pi (mod 2). Since pi ^ P2, we have 
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M v (u ® u) ± M p ,2n+i- P for all p = p\ (mod 2). By Theorem H3J M n (u ® u) = 0. We 
reach a contradiction. We have finished showing that 

n^, 2 „ + i- pl (Mp(n,M))nn2 2i2n+1 _ J)2 (Mp(n,K)) = 0. 

Definition 5.2 Le£ n be a genuine tempered representation of Mp(n,R). -Fza; a parity of 
p. We call (p,2n + 1 — p) the signature of it for p odd (even) if ' ix G n 2 2jl+1 _ p (Afp(n, R)). 
We denote the odd p by p and the even p by p e . We call {p e , 2n + 1 — p e ;p , 2n + 1 — p ) 
the signature ofir. 

From the proof of Theorem 15.31 we obtain 

Corollary 5.1 Fix a parity of p. n G U 2 _(Mp(n,R)) is in H p2n+1 _ p (Mp(n, R)) if and 
only if the K -finite matrix coefficients of tt are all perpendicular to M. pi 2 n +i-p 1 for every 
Pi ^ p with p\ = p (mod 2). 

Notice that any K- finite matrix coefficient f{g) is bounded by a multiple of S(<7) and 
functions in A4 Pl ,2n+i-p 1 are all bounded by A 2rl+1 . So it makes sense to say that 

(/(#), •A'Wn+l-pi) = 0. 

In fact, we can make a stronger statement. 

Corollary 5.2 Fix a parity of p. ir G IPl (Mp(n, R)) is in H p 2n+i-p(Mp(n, R)) if and only 
if there exists a K -finite matrix coefficient f of ' tt and a matrix coefficient F in M. p .2n+i-p 
such that 

I f(g)F(g)dg ? 0. 

JMp(n,I) 

Proof: We prove the only if part by contradiction. Suppose (f(g), M Pl 2n+l-p) — for every 
if-finite matrix coefficient of /. Since tt G H p 2n+1 _ p (Mp(n, R)), by the previous Corollary, 
(/(fl r ))-A4 Pl ,2n+i-p 1 ) = for every p\ ^ p with p\ = p (mod 2). By Theorem 14. 5| / = 
almost everywhere. 7r is not a representation. 

Now suppose that there exists a i^-finite matrix coefficient / of 7r and a matrix coefficient 
F in M Pt 2n+i-p such that 

f(g)F(g)dg ? 0. 



IMp(n,R) 

If 7r ^ lip 2n+1 _ p (Afp(n, R)), then by Theorem 15.31 there exists a p\ ^ p and p\ = p 
(mod 2) such that tt G II* )an +i- pi (Afp(n,M)). By Cor. O (/,M p , 2 „+i- P ) = 0. So 
Jjj , jg-j f{g)F{g)dg = 0. This is a contradiction. Q.E.D. 

Theorem 5.4 Let D be a genuine discrete series representation of Mp(n,M) . Fix a parity 
of p. Then there is a unique p such that D is equivalent to a subrepresentation of LO Pi 2n+i-p- 

Proof: Since D is tempered, D is in the weak closure of Wp^n+i-p for a unique p. Because 
D is isolated in Ii 2 _ (Mp(n, R)), D must occur as a subrepresentation of w p .2n+i-p- Q.E.D. 

Comparing this theorem with Theorem 15.21 we see that Definition 15.21 coincides with 
Definition 15. II for discrete series representations. 



22 



5.3 Howe Duality and A Theorem of Adam and Barbasch 

To define signature beyond tempered representations, we encounter some serious technical 
difficulties. The trouble is that the push forward of the matrix coefficients of a nontempered 
7r may fail to be locally integrable near the boundary with respect to a certain measure. At 
this stage, we do not know how to overcome this obstacle. Nevertheless, we can bypass this 
by relating our results to the Howe duality ( [T4]l. 

Let n(G) denote the admissible dual of a semisimple Lie group G. Let lZ(Mp(n, R), w P) 2n+i-p) 

be those tt in TL(Mp(n,M,)) such that tt occurs as a quotient of w Pi 2n+i-p infinitesimally. 

Howe's theorem then states that there is a one-to-one correspondence between lZ(Mp(n, R), w Pi 2 n +i-p) 

and lZ(0(p, 2n + 1 — p) 1 uo p ^n+i- P )- We denote the Howe duality for (Mp(n, R), 0(p, 2n + 

1 -p)) by 9(2n;p,2n+l - p) and 9(p,2n + 1 -p;2n). 

A result of Adam and Barbasch states that 9(2n;p, 2n + 1 — p) induces a one-to-one corre- 
spondence between n (Mp(n, R)) and the disjoint union of H(SO(p, 2n+l — p)) for p with 
a fixed parity. In particular, n_ (M p(n, R)) is the disjoint union of lZ(Mp(n, R), w Pl 2n+i-p) 
for p with a fixed parity. We can now define a signature of an irreducible genuine represen- 
tation tt to be (p,2n+l — p) if tt 6 TZ(Mp(n, R^u^n+i-p)- Is this definition different from 
ours when tt is tempered? The answer is No. 

We shall now use the theory developed in [7] to show that 

Theorem 5.5 Let tt be a tempered representation, tt € TL p 2n+ i_ p (Mp(n, R)) if and only 
ifn € TZ(Mp(n,M.),uj p> 2n+i-p) (see fT$). 

Proof: Let tt be a genuine tempered representation of Mp(n,M). Suppose that tt 6 
Hp 2n+i-p(Mp(n, R))- Then by Definition 3.2.1 [8], tt is in the semistable range of 8(2n;p, 2n+ 
1 — p). Let / be a X-finite matrix coefficient of tt. By Cor 15.21 (f(g), M p .2n+i- P ) ^ 0. By 
Theorem 1.1 [7|, tt G 1Z(Mp(n,R),L0 Pl2n+1 - p ). 

Conversely, since tt is in the semistable range of 0(2n;p,2n + 1 — p), by Theorem 1.1 
0> (f(d)> -Mp^n+i-p) 7^ for a if -finite matrix coefficient f(g) of tt. By Cor 15.21 
7r G n2 i2n+1 _ p (Mp(n,R)). Q.E.D. 

Finally, let me state a theorem concerning tempered representations of SO(p, q) and Mp(n, R). 

Theorem 5.6 Fix a parity of p. 9 induces a one-to-one correspondence between H 2 _(Mp(n,M.)) 
and Ull 2 (SO{p, 2n + l-p)). 

Proof: Fix a parity of p. We give our proof in two steps. 

First, let tt be a tempered irreducible representation of SO(p, 2n+l—p). By Def. 3.2.1 ( [8]), 
tt is in the semistable range of 9(p, 2n+ 1 — p; 2n). By Theorem [TU], 9(p, 2n+ 1 — p; 2n)(7r) 
exists. By Lemma 6.2.1 and Example 2 ( [8 ), 9(p, 2n + l— p; 2n)(7r) is a tempered irreducible 
representation. 

Second, let tt e II?. (Mp(u,ffi)). Let (p, 2n + 1 — p) be its signature, depending on the 
parity of p. By Def. 3.2.1 ( ^8]), tt is in the semistable range of 9(2n;p, 2n + 1 — p). From 
theorem 15.51 9(2n;p, 2n+l —p)(tt) is nonvanishing. By Lemma 6.3.1 and Example 3 ( [5]), 
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9(2n;p, 2n + 1 — p)(tt) is a tempered irreducible representation. Q.E.D. 
We formulate the following conjecture concerning Howe duality (see 

Conjecture 1 Lei {G\,G2) be a real reductive dual pair. Then tt occurs in 1Z{MG\,<jS) if 
and only if the matrix coefficients of tt can be approximated by the restrictions of matrix 
coefficients of to onto MG\ uniformly on compacta. 

5.4 p + q = 2n + 2: Some Results of Moeglin 

C. Moeglin treats the case p + q = 2n + 2 with p, q both even. One of her results is that 
every tt 6 H(Sp(n,WL)) occurs as a quotient of u) p< 2 n +2—p f° r some even integer p. In this 
section, I shall briefly show that every tempered representation tt occurs as a quotient of 
<^p,2n+2-p for an even integer p e and an odd integer p°. We need the following lemma. 

Lemma 5.1 The linear span of {M p .2n+2-p | P even} is dense in L 2 (Sp(n,M)) and the 
linear span of {Ai Pt 2n+2—p I P odd} is also dense in L 2 (Sp(n,M.)). 

This Lemma follows directly from Theorem 14.41 and Lemma 14.51 The proof is omitted. We 
now have 

Theorem 5.7 Fix a parity of p. Let tt € U 2 (Sp(n, R)). Then n E K(Sp(n, M), uj(p, In + 
2 — p; 2n)) for at least one p. 

Finally, let me make a conjecture concerned with the question about the growth of matrix 
coefficients at oo. 

Conjecture 2 Let tt be a unitary irreducible representation of Sp(n, R) with an integrable 
infinitesimal character. Let M w (u (g) v)(g) be a K -finite matrix coefficient. Then the push 
forward M w {u ® v)(g)° onto S2 n is analytic. 
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